Skip to content

Feature/45 coverage matrix generator service#46

Merged
miroslavpojer merged 4 commits into
masterfrom
feature/45-Coverage-Matrix-Generator-service
Jul 3, 2026
Merged

Feature/45 coverage matrix generator service#46
miroslavpojer merged 4 commits into
masterfrom
feature/45-Coverage-Matrix-Generator-service

Conversation

@miroslavpojer

@miroslavpojer miroslavpojer commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Description

This PR introduces JSON schema validation for the doc-source envelope in the Coverage Matrix Generator service. It adds a new schema definition for doc-source v1.0.0 and implements schema validation logic to ensure all inputs conform to the defined structure before processing.

The changes streamline data consistency across the toolkit and reduce fixture complexity through schema-compliant test data updates.

Changes

  • Added JSON schema for doc-source v1.0.0 (doc-source-v1.0.0-schema.json)
  • Implemented schema validation module (schema_validation.py)
  • Updated collector_gh models and parser for v1.0.0 compatibility
  • Enhanced coverage matrix schema with refined structure
  • Simplified test fixtures and golden files
  • Updated architecture and contracts documentation

Release Notes:

  • Schema validation now enforced on all doc-source inputs before processing
  • Collector adapters updated to handle doc-source v1.0.0 format
  • Test fixtures reorganized with reduced complexity
  • Parser behavior refined for schema compliance
  • Documentation updated to reflect new validation approach

Closes #45

Summary by CodeRabbit

  • New Features

    • Added a new coverage-matrix CLI command to generate coverage reports and enforce an optional minimum coverage threshold.
    • Introduced coverage matrix output support, including a new JSON report format and example usage in the docs.
  • Bug Fixes

    • Updated issue and test data handling to use a more structured user-story format.
    • Improved coverage calculations, including handling of deprecated acceptance criteria and unmatched test references.
  • Documentation

    • Expanded developer, architecture, README, and contract docs for the new coverage workflow and output format.

@miroslavpojer miroslavpojer self-assigned this Jun 30, 2026
@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 57aad984-93df-49c1-80ae-2b5d1f2f70fd

📥 Commits

Reviewing files that changed from the base of the PR and between cde5c76 and 725bb5e.

📒 Files selected for processing (57)
  • .github/workflows/integration.yml
  • .github/workflows/test.yml
  • .gitignore
  • CHANGELOG.md
  • DEVELOPER.md
  • Makefile
  • README.md
  • apps/cli/pyproject.toml
  • apps/cli/src/living_doc_cli/commands/coverage_matrix.py
  • apps/cli/src/living_doc_cli/main.py
  • apps/cli/tests/integration/test_cli_invocation.py
  • apps/cli/tests/test_cli.py
  • docs/architecture.md
  • docs/contracts.md
  • packages/adapters/collector_gh/README.md
  • packages/adapters/collector_gh/schemas/doc-issues-v1.0.0-schema.json
  • packages/adapters/collector_gh/src/living_doc_adapter_collector_gh/models.py
  • packages/adapters/collector_gh/src/living_doc_adapter_collector_gh/parser.py
  • packages/adapters/collector_gh/tests/fixtures/collector_v1.0.0/input/doc-issues.json
  • packages/adapters/collector_gh/tests/fixtures/collector_v1.2.0/input/doc-issues.json
  • packages/adapters/collector_gh/tests/test_models.py
  • packages/adapters/collector_gh/tests/test_parser.py
  • packages/services/coverage_matrix/README.md
  • packages/services/coverage_matrix/pyproject.toml
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/__init__.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/loader.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/matcher.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/model/__init__.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/model/coverage_item.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/schema/coverage-matrix-v1.0.0-schema.json
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/schema/doc-source-v1.0.0-schema.json
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/schema_validation.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/service.py
  • packages/services/coverage_matrix/src/living_doc_service_coverage_matrix/summary.py
  • packages/services/coverage_matrix/tests/__init__.py
  • packages/services/coverage_matrix/tests/fixtures/golden/doc_source.json
  • packages/services/coverage_matrix/tests/fixtures/golden/expected_coverage_matrix.json
  • packages/services/coverage_matrix/tests/fixtures/golden/ui_tests.json
  • packages/services/coverage_matrix/tests/integration/__init__.py
  • packages/services/coverage_matrix/tests/integration/test_golden_files.py
  • packages/services/coverage_matrix/tests/test_loader.py
  • packages/services/coverage_matrix/tests/test_matcher.py
  • packages/services/coverage_matrix/tests/test_service.py
  • packages/services/coverage_matrix/tests/test_summary.py
  • packages/services/normalize_issues/src/living_doc_service_normalize_issues/builder.py
  • packages/services/normalize_issues/src/living_doc_service_normalize_issues/service.py
  • packages/services/normalize_issues/tests/fixtures/golden/expected_output.json
  • packages/services/normalize_issues/tests/fixtures/golden/input.json
  • packages/services/normalize_issues/tests/integration/test_golden_files.py
  • packages/services/normalize_issues/tests/test_builder.py
  • packages/services/normalize_issues/tests/test_service.py
  • tests/fixtures/collector_gh/v0.9.0/input/doc-issues.json
  • tests/fixtures/collector_gh/v1.0.0/input/doc-issues.json
  • tests/fixtures/collector_gh/v1.2.0/input/doc-issues.json
  • tests/fixtures/collector_gh/v2.0.0/input/doc-issues.json
  • tests/fixtures/golden/v1.0.0/expected_output.json
  • tests/fixtures/golden/v1.2.0/expected_output.json

Walkthrough

This PR adds a new coverage_matrix service package that cross-references doc-source and UI-test outputs to produce an AC-level coverage matrix, with its own CLI command, schemas, and tests. It also migrates the collector_gh adapter and normalize_issues service from an items/body schema to a user_stories/description schema across code, fixtures, and documentation.

Changes

Coverage Matrix Service

Layer / File(s) Summary
Build, CI, and package wiring
.github/workflows/*.yml, .gitignore, Makefile, DEVELOPER.md, README.md, apps/cli/pyproject.toml, packages/services/coverage_matrix/pyproject.toml, .../README.md, .../__init__.py
Adds coverage_matrix install steps to CI/Makefile/dev docs, package metadata, and README documenting inputs/outputs and module layout.
CLI command entrypoint
apps/cli/src/living_doc_cli/commands/coverage_matrix.py, apps/cli/src/living_doc_cli/main.py, apps/cli/tests/test_cli.py
Adds the coverage-matrix Click command with required/optional flags, error handling, registration, and tests.
Loading & doc-source validation
.../loader.py, .../schema_validation.py, .../schema/doc-source-v1.0.0-schema.json, tests/test_loader.py
Loads doc/tests JSON inputs and validates doc-source envelopes against the new schema.
Matcher, model, and output schema
.../matcher.py, .../model/coverage_item.py, .../schema/coverage-matrix-v1.0.0-schema.json, tests/test_matcher.py
Resolves test scenarios into user story/functionality coverage, unlinked tests, and stale references, backed by dataclasses and JSON schema.
Summary computation
.../summary.py, tests/test_summary.py
Computes per-entity and top-level coverage percentages and totals.
Service orchestration
.../service.py, tests/test_service.py
Orchestrates loading, filtering, matrix building, output writing, and fail_under threshold enforcement.
Golden fixtures & integration tests
tests/fixtures/golden/*, tests/integration/test_golden_files.py
Validates the end-to-end pipeline against fixed input/output fixtures.
Architecture & contracts docs
docs/architecture.md, docs/contracts.md
Documents the new service's pipeline position, components, CLI contract, and output schema.

User Story Schema Migration

Layer / File(s) Summary
Collector-gh schema and models
packages/adapters/collector_gh/schemas/*.json, .../models.py, .../README.md
Renames items/body to user_stories/description and adds structured fields.
Collector-gh parser
.../parser.py
Parses and validates the new user_stories payload structure directly.
Collector-gh tests & fixtures
.../tests/*
Updates unit tests and input fixtures to the new schema.
Normalize-issues builder & service
packages/services/normalize_issues/src/.../builder.py, .../service.py
Consumes user_stories and structured fields directly, removing markdown parsing.
Normalize-issues tests & fixtures
packages/services/normalize_issues/tests/*
Updates tests and golden fixtures to structured sections.
Repo-level collector fixtures & golden outputs
tests/fixtures/collector_gh/*, tests/fixtures/golden/*
Updates versioned collector fixtures and expected outputs to the new schema.
CLI integration test fixture
apps/cli/tests/integration/test_cli_invocation.py
Updates the CLI integration fixture to the new user-story structure.

Estimated code review effort: 4 (Complex) | ~75 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant CoverageMatrixCLI
  participant Loader
  participant Matcher
  participant SummaryModule
  participant OutputFile

  User->>CoverageMatrixCLI: coverage-matrix --doc-input --tests-input --output
  CoverageMatrixCLI->>Loader: load_doc_input / load_tests_input
  Loader-->>CoverageMatrixCLI: doc groups, test items
  CoverageMatrixCLI->>Matcher: build_coverage_matrix(doc, tests)
  Matcher->>SummaryModule: compute_us_summary / compute_summary
  SummaryModule-->>Matcher: Summary / TopSummary
  Matcher-->>CoverageMatrixCLI: CoverageMatrix
  CoverageMatrixCLI->>OutputFile: write coverage-matrix.json
  CoverageMatrixCLI-->>User: success or fail_under error
Loading

Possibly related PRs

  • AbsaOSS/living-doc-toolkit#41: Both PRs modify the same collector_gh models/parser and normalize_issues builder/tests to handle acceptance_criteria and related metadata shapes.

Suggested reviewers: tmikula-dev

Poem

A rabbit hopped through code so bright,
Matching stories with tests just right,
Coverage percents now clear as day,
Stale refs and gaps swept away,
Hop, hop, hooray — the matrix takes flight! 🐇📊

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/45-Coverage-Matrix-Generator-service

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

- Introduced a new JSON schema file for doc-source v1.0.0 to define the structure and requirements for the doc-source envelope.
- Implemented schema validation in `schema_validation.py` to ensure that inputs conform to the defined schema before processing.
- Added functions to validate the doc-source envelope and handle validation errors appropriately.
@miroslavpojer miroslavpojer marked this pull request as ready for review July 3, 2026 09:04
@miroslavpojer miroslavpojer merged commit 2c5b0ff into master Jul 3, 2026
4 of 5 checks passed
@miroslavpojer miroslavpojer deleted the feature/45-Coverage-Matrix-Generator-service branch July 3, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: Coverage Matrix Generator service

1 participant